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Abstract — Introduction: Currently, the scientific and technological development has modified several areas of knowledge, 
including the health sciences. One such technology are the Serious Games, which are educational games that focus on specific 
and intentional learning to achieve efective, measurable, and continuous changes in perfomiance and behavior. Ideally, 
students should receive enough training during dental school to have the skills necessary to make a correct diagnosis, and 
perform an effective and safe treatment. The authors believe that serious games can be applied in Endodontics, more 
specifically in emergency care. This study had the objective of developing and validating scientific (Endodontics) and 
pedagogically a cpiiz-type serious game within the theme "Endodontic Emergencies". Materials and Methods: The items on 
"Endodontic Emergencies" were validated by three endodontists (scientific validation) and peiformed with the support of the 
Pedagogical Center of the Open University of the Unified Health System (pedagogical validation). Results: Sixty evaluative 
items on the theme were elaborated and validated based on textbooks and scientific publications in Endodontics. The game 
has several motivational tools and strategies. The game also has moving characters so it is visually dynamic. Conclusion: The 
game was developed within the theme "Endodontic Emergencies" and its questions were validated for content (scientific 
validation) and pedagogically. The game is freely available as an application for the iOS system. From thi s initiative, the 
serious games learning approach is expected to be applied in a larger scale, since its spreading potential and social impact 
have been emphasized. 
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I. INTRODUCTION 

Scientific and technological development has 
modified several areas of knowledge, including health 
sciences. Digital information and communication 
technologies (DICTs) have been increasingly connected to 
health care, with wide acceptance and incorporation by the 
public, health professionals, and educational institutions *. 

DICTs’ inclusion in teaching has become essential for 
students’ critical thinking evolution 2 . 

One of these technologies is Serious Games. Its 
main feature is to support the player to achieve learning 
goals through a fun experience 3 . The fun aspect is to 
encompass graphics, interaction devices, collaboration 
mechanisms and usability, while the learning aspect uses 
the pedagogical approach through educational content 4 . 

Games using multiple-choice questions can be used so that 
players can improve their knowledge about a particular 


topic, since to overcome it you must respond correctly and 
continuously each one of them 5 . 

For question items’ elaboration, we can use the 
Bloom’s Taxonomy in order to align the educational goals 
with the students’ cognitive domain 6 . It was revised in 
2001 and the objectives are encompassed in six categories: 
meet, understand, apply, analyze, synthesize and evaluate 

7 

In oral health, the dental surgeon deals with 
emergency situations forpain relief, since 90% of orofacial 
pain’s cases are related to pulpal and/or periapical causes 
8 . Knowing that emergencies are a reality in the daily life 
of clinical service, the dental surgeon should be able to 
perform them, however, students refer to Endodontics as 
the greatest difficulty area to perform the procedures and 
they feel a high level of anxiety when they're about to do it 

9 
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It is expected that the undergraduate program has 
the ability to provide to professionals in training the 
necessary skills to perform a correct diagnosis, 
implementing a treatment plan and a qualified and safe 
treatment. Competency-based programs can replace the 
traditional dental education methodology, and the aim of 
this modality is described as understanding skills and 
professional values required of a student that are essential 
for beginning the unsupervised practice of dentistry 10 . 

However, we know that this is not always the 
reality, and knowledge democratization by means of 
technological resources is a powerful tool to update and 
in form professionals, and this is where the serious games 
fit. The use of games in contexts that do not normally 
involve game playing is called ‘gamificatioiT, a 
methodology that employs playful and competitive 
elements in real life situations n ’ 12 . 

A review by Azevedo et al. 13 showed that most 
serious games in the Dentistry field are aimed at patient 
distraction and anxiety reduction methods or related to 
clinical simulations. Studies about the effectiveness of 
games for diagnosis and establishment of appropriate 
procedures, especially in Endodontics, are not known. 


We believe that serious games can be applied in 
Endodontics, more specifically in emergency care. This 
way, the objective ofthis study is to suggesta methodology 
for scientific and pedagogical validation of questions to be 
used in quiz-type serious games such as "Endodontic 
Emergencies". 

II. MATERIALS AND METHODS 
2.1 The Game 

This is an educational and interactive game, using 
a 2D interface, enabling an environment of learning about 
"Endodontic Emergencies". The game was developed in 
partnership between the Open University of the Unified 
Health System - UNA-SUS/UFMA (Brazil), the schoolof 
Dentistry of the University of Sao Paulo (Brazil) and the 
Federal University of Maranhao (Brazil). 

The game was idealized to dentistry students (as the issues 
encompass the Endodontics area). The game features 
different motivation tools and strategies such as: tutorial, 
punctuation, response time, opportunity to get tips and 
ranking, content of texts, images and sound, with 
characters in motion, so that it stays dynamic. The Figure 
1 shows the game in 04 scenes that were developed to 
characterize it. 



Fig.l: Scenes from the game quiz development flow. 


Figure A shows the entry that is the showcase of 
the game and the player can access it, showing the credits 
and exit; Figure B, displays character selection; Figure C, 
presents the scene of the gameplay that is where the 
interaction occurs with the focus of the game; and Figure 
D reveals the completion scene that shows the final 
ranking. 


2.2 Didactic-pedagogical project elaboration. 

The game quiz is a serious game because it is 
based on the pedagogical principles focused on the 
construction of skills and abilities. Several technical- 
scientific and pedagogical aspects are taken into account 
and therefore need a prior planning that encompasses the 
several areas of knowledge previously described. 
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It was necessary the development of a 
Pedagogical Workshop test question construction that 
would compose the database of the game. This workshop 
was held in partnership with UNA-SUS/UFMA in the 
months of March and June of 2016, so that the issues were 
pedagogically grounded, resulting in adequate educational 
resources. 

The questions were elaborated following 
recommendations for elaboration of multiple choice 
questions 14 . 

Due to the importance of didactic-pedagogical 
planning, it was necessary to define amain thematic area, 
quantity of questions, its educational objectives, contents 
to be worked, target audience, partner institutions, besides 
other components of the items of evaluation such as the 
statement, alternatives, feedback for each alternative 
(correct or not), the tip to help the player and bibliographic 
reference link. 

2.2.1 Definition of questions number on the theme 
"Endodontic Emergencies". 

Sixty test questions were elaborated through 
bibliographical research in books and scientific articles of 
Endodontics. They were "multiple choice" type and 
contained one statement, five alternatives (one correct), 
each with its feedback or justification. In addition, the 
question has a "tip" that can be requested by the participant 
during the game, and the bibliographic reference used to 
constmct the question. There is also the possibility of 
providing links on the topic involved in the statement. If 
the player makes a mistake, the game will not end, it will 
go on. They will score points for each question, and may 
have a final score of 100 points. 

2.2.2 Educational objectives and definition of difficulty 
levels of questions 

Educational objectives for the serious game in 
"Endodontic Emergency" construction were designed to 
contribute in an interactive and playful way in dental 
practice, so that users can understand and rethink their 
work processes with focus on emergency care. 

During the construction of the testquestions, we 
used cognitive domains inserted in the Bloom Taxonomy 7 . 
The issues considered at the "Easy" level are those that are 
within the cognitive domain "Know" and involve 
memorization of specific facts, patterns of procedure and 
concepts. The "Medium" level questions are those in the 
"Understanding" domain and involve meanings, 
translation and interpretation of problems and instmctions. 
Finally, the "Difficult" questions are those of the 
"Analyze" domain and are more complex because they 
deal with analisis of contexts, elements, relationships and 
principles of organization. Twenty questions were 
elaborated for each level of difficulty, with the level of 
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"Easy" with a time of 45 seconds for answer, the 
"Medium" level with 1 minute for answer and the 
“Difficult” level with 1 min and 15 seconds for answer. 

2.2.3 Content covered 

The main theme is "Endodontic Emergencies", 
but five important sub-themes were chosen for this 
approach. Subjects related to routine dental care were 
selected, but focused on issues that deal with the 
emergency procedure, that is, problems that any dentist 
should be prepared to solve. The chosen subtopics were: 
Endodontic Diagnosis, Radiology applied to 
Endodontics, Anesthesia and Pharmacology, 
Characteristics of Materials, Anatomy and Technical 
Procedures. 

2.3 Evaluative testquestions and scientific and 
pedagogical validations 

2.3.1 Scientific Evaluation 

The test questions evaluation on "Endodontic 
Emergencies" was done by three (03) reference 
professionals in Endodontics with expertise and credentials 
to perform this task (three professors in Endodontics,from 
three different institutions of higher education with 
Dentistry courses). The questions were evaluated 
according to their content in the chosen theme. The number 
of evaluators was based on previous work found in the 
literature 15 ’ 16 . 

For the accomplishment of scientific evaluation, 
the professionals received the questions and added 
comments and suggestions. The data obtained were 
distributed in a table adapted to each question 16 and the 
items were evaluated according to textual clarity, practical 
relevance and suitability to the public. Each category was 
assessed and thereafter considered adequate, partially 
adequate or inadequate, hr addition, there was the 
possibility of adding comments and suggestions to change 
or even discard the issue after the returns. 

2.3.2 Pedagogical Evaluation 

The pedagogical evaluation was carried out by 
experts of the Pedagogical Nucleus of the Open University 
of the Unified Health System of the Federal University of 
Maranhao. These experts received the elaborated 
evaluation test questions and suggested changes. The data 
obtained were also distributed in a table adapted to better 
visualize the results 16 , and it evaluated the categories: 
number of characters, cognitive domain and textual clarity. 

2.4 Database 

After pedagogical and scientific evaluation, the 
testquestions were registered in an online platform form 
called SigU Activities. This platfomr is a questioning aid 
subsystem that allows the insertion of questions, answers 
and feedbacks and represents the optimization of a learning 
object to assist in continuing education in health 17 . 
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III. RESULTS 

3.1 Scientific Evaluation 

After questions analysis and evaluation for such stage, the data were distributed in a table adapted for better visualization 
of the results 16 . Table 1 shows the results of each evaluator. 

Table 1 - Data obtained by the evaluators in scientific validation 


Evaluator 1 Evaluator 2 Evaluator 3 



Textual 

Clarity 

Practical 

Relevance 

Public 

Adequacy 

Textual 

Clarity 

Practical 

Relevance 

Public 

Adequacy 

Textual 

Clarity 

Practical 

Relevance 

Public 

Adequacy 

Adequate 

58.3% 

100% 

100% 

56,6% 

100% 

100% 

80% 

98,3% 

96,6% 

Partially 

adequate 

36,6% 

0 

0 

38,3% 

0 

0 

20% 

1,7% 

3,4% 

Inadequate 

5% 

0 

0 

5% 

0 

0 

0 

0 

0 


The results maintained a consistency between the 
first two experts. All test questions were considered 
appropriate with respect to the practical relevance and 
adequacy to the public for them, but the third expert 
encountered some issues with interpretation difficulties 
related to these two items. Thus, the initial definition on the 
subject addressed in each evaluation was improved, along 
with adjustments in the writing of the utterance, according 
to the evaluator's suggestions. 

The most important finding was with regard to 
textual clarity. For the first expert, 58.3% of the questions 
were considered adequate, for the second, 56.6% and for 
the third, 80% in the same category. For the first expert, 
36.6% of the questions were considered partially adequate. 


38.3% for the second and 20% for the third. The first two 
experts considered 5% of the questions inadequate. 

The issues considered inadequate had to be 
completely modified, as they presented some kind of 
ambiguity in their content. The partially adequate ones 
obtained suggestions of improvements in the writing to 
facilitate the understanding of the player and the adequate 
ones did not need any changes. 

3.2 Pedagogical Evaluation 

The Pedagogical Nucleus of UNA-SUS/UFMA 
validated the questions regarding didactic-pedagogical 
aspects and this evaluation was carried out in pairs. The 
data were compiled in a table adapted to better visualize 
the results 16 . Table 2 shows the results obtained. 


Table 2 - Data obtained by pedagogical validation 



Number of characters 

Cognitive Domain 

Textual Clarity 

Adequate 

51,6% 

66,6% 

11,6% 

Partially adequate 

13,3% 

1,6% 

85% 

Inadequate 

35% 

31,6% 

3,3% 


The questions considered partially adequate 
(13.3%) were within the limits required by the SigU 
Activities platform, some alternatives were in disarray in 
size, it was considered a heuristic in game. Therefore, the 
questions should be modified. The test questions 
considered to be inadequate had the number of characters 
outside the SigU Platform boundaries and limits and 
should be modified. 

Only one question (1.6%) was considered 
partially adequate, meaning that the level of difficulty it 
was correct, but the type of domain covered was not, so the 
question was modified. 

The test questions considered inadequate (31.6%) 
had to be reinterpreted in the level of difficulty. That is, 
initially 20 questions were obtained for each level of 
difficulty and after the pedagogical validation there were 


22 questions of the "Easy" level, 24 of the "Medium" level 
and 14 of the "Difficult". 

The test questions considered inadequate (3.3%) 
were elaborated using negative expressions such as NO or 
EXCEPT (distractors), and this fact should be avoided, 
since it impairs the player's understanding and resolution. 
Thus, the issues needed to be completely changed, from the 
statement to the alternatives. 

IV. DISCUSSION 

The test questions for the quiz game "Endodontic 
Emergencies" were evaluated by pedagogical and 
endodontics experts, thus suggesting this methodology as 
a model for new serious games. Bloom’s Taxonomy 
purposes have been successfully followed overtime in a 
variety of curricula, disciplines, and contexts, including the 
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field of health science education. Recent studies in the 
health field have been published involving the construction 
of question items (multiple choice) based on Bloom’s 
Taxonomy 18 , 19 , 20,21 

The teaching performance should understand the 
levels found in the Taxonomy of Bloom which in 
Dentistry are practically encompassed only those related to 
the cognitive domains 22 , among them "to know", "to 
understand" and "to analyze" that were used in this work. 

Based on the selected theme, it is possible to note 
the practical relevance, since pain of dental origin 
interferes in the quality of life of the patients, since it 
causes social and professional damages such as sleep 
disturbances,physical and emotional fatigue, loss of social 
contact, food, limitation of daily activities, and 
absenteeism at work 23 . 

Learning motivation is an important element in 
students education. The use of technology is facilitated by 
the upcoming of the digital era, and the easy access to 
various resources and media 24 . hi addition, quiz-type 
educational games can be based on previous knowledge 25 
and the game presented herein provides feedbacks in each 
response with additional information for the player, 
enriching the learning experience. 

The search for knowledge is a widespread reality 
today. For the new generations, the access to technologies 
is part of everyday life, and this way, the motivation for 
using technology in education can increase meaningful 
learning. Consequently, game playing using this resource 
while, above all, enriching the learning experience can be 
stimulating, especially to students used to traditional 
methods of teaching, which often lack major innovations 
26>27 . A great advantage of learning through games is the 
reinforcement of intrinsic motivation and engagement, 
which can generate opportunities for further knowledge 
seeking 28 - 29 . 

The construction of educational strategies based 
on serious games is feasible, as long as it has the support 
of professionals from the areas of knowledge needed to 
reach the competencies and abilities defined for the theme. 
The pedagogical support of experts who validate the 
knowledge involved and the adequacy of the content 
guarantees the appropriateness of serious game learning 
strategies. 

With the game "Endodontic Emergencies", we 
expect that the users can test their knowledge, and thus be 
induced to seek further learning that is one of the main 
objectives of continuing education. This component is 
promoted by the instigation resulting from the game 
playing, besides the access to links and bibliographical 
references that can provide support for the search of 
knowledge. 
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The use of serious games has a fertile ground in 
formal education, as students are already inserted in a 
social context surrounded by media and technologies. The 
strategy can provide new and different ways of teaching 
and learning than the passive traditional methods 30 . 

As a limitation, the level of difficulty of the game 
questions was a subjective definition, since it depended on 
each player involved, hi addition, the work of Diab and 
Sartawi 31 has shown that there is an expressive use in the 
academic environment of Bloom's Taxonomy for 
questioning, but suggests that there is an ambiguity behind 
this classification and a need for more accurate verbs that 
aid in mental acuity. 

hi this perspective, the UNA-SUS / UFMA has 
planned a more objective evaluation through the game's 
analytics that will identify, for example, which questions 
were more often answered correctly overtime, and thus be 
adjusted as “easy”, accordingly. Similar analysis could be 
done to classify the “intermediate” and “difficult” items. 

The use of serious games has grown and it is 
necessary that this approach is increasingly exploited, 
since we are inserted in a society in which scientific and 
technological advances go hand in hand. More research is 
needed to carry out a more complete assessment of the 
acceptance and verification that the use of serious games 
increases the effectiveness of learning training. 

V. CONCLUSION 

The test questions for the quiz game "Endodontic 
Emergencies" were evaluated by experts, thus suggesting 
this methodology as a model for future quiz games. The 
game was made available in the form of application to the 
Android system. This way, the user has easy access and is 
free to play. From this initiative, the serious games learning 
approach is expected to be applied in a larger scale, since 
its spreading potential and social impact have been 
emphasized. 
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